Accelerated Point-Wise Maximum Approach to Approximate Dynamic Programming

In this article, we describe an approximate dynamic programming (ADP) approach to compute lower bounds on the optimal value function for a discrete time, continuous space, and infinite horizon setting. The iteratively constructs family of bounding functions by using so-called Bellman inequality. novelty our is that, at each iteration, aim that maximizes point-wise maximum taken with computed thus far. This leads nonconvex objective, propose gradient ascent algorithm find stationary points solving sequence convex optimization problems. We provide convergence guarantees interpretation how computation relates state-relevance weighting parameter appearing in related ADP approaches. demonstrate through numerical examples when compared existing approaches, computes tighter suboptimality comparable time.